Persistent memory (PM) enabled operating systems (OSs) include a direct access (DAX) mechanism to expose PM media to applications with direct memory mappings. “Direct” in this context means no indirection through an OS buffer cache, and load/store instruction data access from the processor (that is, no intermediary host adapter). This minimizes sources of overhead in the data persistence path, but also compromises the ability of the OS to perform address space management operations.
In particular, an OS filesystem that has made a DAX mapping available to device direct memory access (DMA) may determine the need to remap that access while the access operation is in process. Consider a filesystem that is directed to snapshot the current state of a file. With indirection the filesystem can buffer ongoing DMA in memory and write the data later, but without that indirection the only option is to wait for a quiescent state for device DMA before the device DMA can be redirected to a new physical address (e.g., a new DAX page).
For DAX, if the only solution for redirection is to wait for active device DMA to quiesce this can lead to unbounded and indefinite waits. While most DMA transactions are transient, remote direct memory access (RDMA) and other protocols may keep a DMA operation active indefinitely at the control of the RDMA application, but not the OS (that is, not the filesystem that is the owner of the address space layout for a file). As DAX applications proliferate and the number of input/output (I/O) devices and offload devices increases, collisions between address space management operations and device DMA operations will likely increase, negatively impacting system performance.